Development of Methods for Usability Evaluations of EHR Systems

Developing electronic health record (EHR) systems in Denmark is an on going, iterative process, where also a maturation process for clinical use should be considered. Convincing methodology for collecting and incorporating in the soft- and hardware knowledge and robustness for the clinical environments is not on hand. A way to involve the clinicians in the development process is conducting usability evaluations. The complexity of the clinical use of the systems is difficult to transmit to a usability laboratory, and due to ethical issues a traditional field study can be impossible to carry out. The aim of this study has been to investigate how it is possible to identify usability problems in an EHR system by combining methods from laboratory tests and field studies. The methods selected for the test design are: the think aloud method, video and screen recording, debriefing, a scenario based on an authentic patient record, and testing on the normal production system. The reliability and validity of the results is increased due to the application of method- and data-triangulation. The results of the usability evaluation include problems in the categories: system response time, GUI-design, functionality, procedures, and error messages. The problems were classified as cosmetic, severe, or critical according to a rating scale. The experience with each method is discussed. It is concluded that combining methods from laboratory test and field study makes it possible to identify usability problems. There are indications that some of the usability problems only occurred due to the establishment of an authentic scenario.